Data mining explores detailed business transactions to uncover patterns and relationships contained within a particular business activity and history. Data mining can be done manually by “slicing and dicing” the data until a pattern becomes obvious. A term of art in the field, “slicing and dicing” refers to the ability to move between different dimensions of warehoused data. It can be performed with programs that analyze the data automatically. Using computer technology to look for hidden patterns in a collection of data, data mining for marketing research, for example, might reveal that customers interested in one product will also be interested in another. In other areas, data mining can be useful in scientific research, economics, criminology, and many other fields. In general, there exists specialized database software for data mining.
A data warehouse is a database designed to support decision making in an organization. It is batch updated and can contain enormous amounts of data. Hence, the moniker “warehouse”. For example, large retail organizations can have 100 GB or more of transaction history. When the database is organized for one department or function, it is often called a “data mart” rather than a data warehouse.
The data in a data warehouse is typically historical and static and may also contain numerous summaries. It is structured to support a variety of analyses, including elaborate queries on large amounts of data that can require extensive searching. When databases are set up for queries on daily transactions, they are often known as operational data stores (ODSs) rather than data warehouses.
At present, there exist no data mining services relating to industrial applications that provide data mining and data mart services as described above. Until now, data services for industrial applications have been offered by consultants who manually compile and audit data on a case-by-case basis. Such manual compilation may be inadequate to provide a complete or accurate analysis of the industry being researched. Manual accounting methods, moreover, cannot provide a continuous or virtually continuous update of the industry and are subject to human error.